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ABSTRACT 


A  study  was  conducted  to  assess  the  performance  benefits  associated  with 
the  use  of  the  General  Electric  Engine  Monitoring  and  Display  System 
(EMADS).  An  experiment  was  designed  to  compare  pilot  performance  using 
both  EMAOS  and  conventional  engine  instruments.  Data  was  collected 
from  eight  DC- 10  qualified  pilots  having  an  average  of  over  13,000  hours 
of  flight  experience. 

The  flight  task  for  each  of  the  sixteen  test  trials  consisted  of  engine 
startup,  manual  takeoff,  and  cllmbout  to  5,000  feet  in  a  fixed  base 
simulator.  In  addition,  a  number  of  predefined  engine  related  fault 
conditions  were  Introduced  at  various  points  during  the  simulation, 
with  the  pilots  being  Instructed  to  execute  the  appropriate  corrective 
action. 


For  each  of  the  fault  monitoring  tasks,  performance  (reaction  time)  with 
EMADS  was  as  good  or  better  than  that  resulting  from  the  use  of 
conventional  Instruments.  Subjective  input  obtained  from  the  pilots 
Indicated  a  clear  preference  for  EMADS.  Reconmendatlons  are  made 


regarding  future  research  activity. 
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INTRODUCTION 

The  continuing  advancement  of  high-performance  aircraft  has  brought  with 
it  an  increasing  degree  of  complexity  in  the  physical  operating  system 
due  to  such  factors  as  flight  operation  complexity,  demands  for  improved 
safety,  improved  reliability,  and  more  efficient  operations.  Such 
demands  require  the  development  of  advanced  concepts  for  status 
monitoring  and  information  displays  for  aircraft  systems.  The 
excessive  number  of  indicators  and  annunciators  in  the  cockpit  is 
rapidly  approaching  the  saturation  point  in  terms  of  crew  capability  to 
assimilate  and  assess  the  significance  of  indications,  make  decisions, 
and  take  appropriate  action.  Besides  adding  to  crew  workload,  this 
proliferation  of  discrete  annunciators  Is  reducing  the  availability  of 
installation  space  for  equipment  in  the  cockpit. 

The  development  of  advanced  digital  display  technology  for  flight  deck 
applications  has  received  considerable  attention  In  recent  years. 
Manufacturers  of  aircraft  and  avionics  equipment  have  allocated 
substantial  research  and  development  efforts  toward  improving  cockpit 
efficiency  through  the  use  of  electronic  display  media.  In  response  to 
this  need,  a  joint  research  effort  was  undertaken  by  General  Electric 
(GE)  and  Douglas  Aircraft  Company  (DAC)  to  develop  an  Engine  Monitoring 
and  Display  System  (EMADS).  EMADS  has  been  advanced  as  a  viable 
alternative  to  the  conventional  electromechanical  engine  instruments 
presently  being  used  In  comnercial  aircraft.  The  specific  purpose  of 
EMADS  is  to  provide  the  flight  crew  with  displays  for  all  engine 
management  activities.  It  Is  also  designed  to  provide  engine 
maintenance  and  performance  summaries  for  post-flight  analysis. 

Summary  of  Previous  Developmental  Efforts 

The  EMADS  program  was  initiated  in  1974.  In  1975,  system  operating 
concepts  and  display  requirements  were  evaluated  and  refined.  The 
results  were  developed  Into  system  scenario  formats  for  programming  in 
the  Digital  Equipment  and  Technology  Analysis  Center  (DETAC),  located 
at  Douglas  Aircraft  Company  in  Long  Beach,  California.  Flow  diagrams 
were  completed  and  updated  during  the  first  quarter  of  1976  and  were 

1 


then  used  as  the  basis  for  development  of  functional  system  requirements, 
display  and  input  requirements  for  continuous  and  discrete  parameters. 
Completion  of  the  DETAC  simulation  in  1977  provided  candidate  display 
formats,  mode  definition,  and  verification/refinement  of  crew-system 
interfaces.  These  results  were  then  used  by  GE  for  software  development 
and  final  completion  of  the  hardware  development  and  fabrication.  A 
parallel  effort  was  continued  by  GE  to  develop  the  engine  model  which 
was  to  be  programmed  into  the  Motion  Base  Simulator  (MBS)  computer. 

After  the  system  was  received,  Installed,  and  checked  out  on  the  MBS, 

s' 

ten  DAC  pilots  participated  in  simulation  runs  using  the  procedure 
developed  by  DAC  Human  Factors  Engineering.  Results  of  these  initial 
runs  were  used  to  modify  the  system  software.  Twenty  additional 
simulation  runs  were  performed  using  test  pilots  and  flight  engineers 
from  DAC.  Each  pilot  flew  a  partial  flight  scenario  Intended  to 
exercise  the  major  functions  of  EMADS.  Following  simulation  trials, 
each  subject  participated  In  a  debriefing  session  and  completed  a 
comprehensive  questionnaire  evaluating  EMADS  operational  characteristics. 
Based  on  the  judgments  of  these  experienced  pilots,  EMADS  offers  the 
potential  for  significant  reduction  in  cockpit  workload  from  the  effort 
required  with  conventional  engine  instruments. 

In  1978,  EMADS  was  evaluated  by  representatives  from  various  airlines, 
military  services,  and  regulatory  agencies.  A  total  of  ninteen 
organizations  participated  in  the  demonstration  which  was  conducted  in 
the  DAC  MBS.  The  scenario  provided  an  opportunity  for  "hands-on" 
familiarization  with  the  EMADS  control  and  display  unit  and  demonstrated 
the  full  range  of  EMADS  operational  capabilities.  Following  the 
simulation,  pilots  viewed  a  demonstration  of  potential  color-coding 
applications  to  the  baseline  monochromatic  EMADS  display  formats.  The 
overall  pilot  evaluation  of  the  system  was  quite  positive.  Sixty-three 
percent  (63%)  of  the  pilots  said  that  EMADS  was  superior  to  conventional 
dials.  Another  major  strength  of  the  EMADS  system  is  Its  flexibility. 
Eight-three  percent  (83%)  of  customer  pilots  indicated  that  the  design 
and  operation  of  the  system  would  readily  accommodate  changes  in 
procedures  and  differences  in  operating  methods  across  airlines. 
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In  1979,  Douglas  Aircraft  sponsored  a  flight  operations  seminar  at  which 
EMADS  was  demonstrated  to  approximately  one  hundred  forty-five  pilots  and 
other  flight  operations  personnel.  A  total  of  twenty-two  of  these  indi¬ 
viduals  completed  and  returned  a  questionnaire  that  addressed  the  relative 
merits  of  EMADS  and  conventional  engine  instruments.  The  respondents 
clearly  felt  that  all  rated  EMADS  features  could  reduce  cockpit  workload 
compared  to  conventional  instruments.  They  also  felt  that  EMADS  wouid 
be  more  effective  in  facilitating  quick  and  accurate  responses  to  fault 
conditions.  In  general,  they  were  quite  impressed  by  the  EMADS 
display  characteristics,  rating  them  from  excellent  to  better  than 
average  in  terms  of  brightness,  contrast,  display  format,  legibility, 
etc.  About  twenty-five  percent  (25%)  of  the  respondents  expected  no 
significant  transition  problems  between  EMADS  and  conventional  instru¬ 
ments.  The  remainder  of  the  participants  expected  at  least  some 
transition  problems  .possibly  during  initial  training.  Overall,  reaction 
to  this  demonstration  was  positive,  although  only  fifteen  percent  (15%) 
of  the  observers  completed  and  returned  the  questionnaire. 

During  the  EMADS  development  program,  a  number  of  color  coding  appli¬ 
cations  have  been  suggested,  some  of  which  have  been  evaluated  as  part 
of  the  demonstration  mentioned  earlier.  In  general,  participants 
reacted  favorably  to  the  use  of  color  in  the  EMADS  displays.  As  with 
any  information  display,  color  can  provide  a  useful  dimension  if  it  is 
applied  appropriately.  If  color  is  to  be  incorporated  into  the  EMADS 
information  display,  it  should  be  carried  out  in  concert  with  existing 
guidelines  for  the  applications  of  color  to  alert  urgency  levels  (Berson, 
Po-Chedley,  Boucek,  Hanson,  Leffler,  and  Wasson,  1981;  Society  of 
Automative  Engineers,  1980). 


Study  Objective  and  Experimental  Hypotheses 

The  primary  purpose  of  the  present  study  was  to  provide  an  objective 
evaluation  of  EMADS  and  conventional  instruments.  Several  key  questions 
were  addressed  in  order  to  determine  if  EMADS  is  as  good  as  or  better 
than  the  present  system.  A  comparison  was  made  to  determine  pilot 
performance  characteristics  using  EMADS  and  conventional  instruments 
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during  normal  operations  and  also  in  identifying  and  correcting  faults 
during  various  flight  phases.  While  all  areas  of  aircraft  operations 
were  investigated,  the  focus  of  this  study  was  directed  toward 
monitoring  preflight,  takeoff,  and  inflight  operations. 

Both  subjective  and  objective  data  were  collected.  Subjective  data  was 
acquired  by  means  of  a  questionnaire  administered  after  completion  of  the 
test  profile.  Objective  performance  data  was  also  collected  to  compare 
EMADS  and  conventional  instruments  under  controlled  experimental  treat¬ 
ment  conditions.  In  preflight,  under  normal  conditions,  response  times 
were  recorded  in  terms  of  total  time  to  start  engines.  Both  detection 
and  response  times  were  collected  for  preflight  fault  monitoring 
activity.  For  the  normal  operations  portion  of  the  takeoff  flight 
segment,  performance  level  was  determined  by  the  pilot’s  ability  to 
accurately  set  the  throttles  to  the  appropriate  N-j  command  value.  After 
takeoff  and  during  the  climb  segment  of  the  flight,  faults  were 
introduced  to  determine  which  display  system  is  most  conducive  to  optimal 
pilot  performance.  Auxiliary  faults  were  also  incorporated  during 
various  segments  of  the  flight  to  extend  the  pilots  point  of  focus.  A 
detailed  description  of  each  of  these  treatment  conditions  is  contained 
in  the  following  section  of  this  report.  From  the  questions  that  have 
been  raised  in  the  preceding  section  the  following  hypotheses  were 
derived: 


Hypothesis  1  -  EMADS  will  be  as  good  as  or  better  than  conventional 

instruments  in  minimizing  response  time  for  normal  start 
procedures . 

Hypothesis  2  -  EMADS  will  be  as  good  as  or  better  than  conventional 

Instruments  on  throttle  setting  accuracy  relative  to  the 
mean  value  at  80  knots  and  at  under  normal  takeoff 
operations. 

Hypothesis  3  -  EMADS  will  be  as  good  as  or  better  than  conventional 

instruments  In  terms  of  time  required  for  fault  detection. 

r.vy  Hypothesis  4  -  EMADS  will  be  as  good  as  or  better  than  conventional 
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instruments  in  terms  of  total  response  time  required  for 
fault  detection. 


Hypothesis  5  -  EMADS  will  be  as  good  as  or  better  than  conventional 

instruments  in  terms  of  fuel  expenditure  during  takeoff 
normal  operations. 

Hypothesis  6  -  EMADS  will  be  as  good  as  or  better  than  conventional 
instruments  in  terms  of  flight  task  performance. 


METHOD 


Pilot  Sample 

A  total  of  nine  pilots  participated  in  the  study,  having  an  average  of 
13,875  hours  of  flight  experience.  American,  Continental,  Western,  and 
United  Airlines  were  each  represented  by  a  management  level  pilot  with 
the  remainder  representing  the  Douglas  flight  test  organization.  To 
maximize  the  effectiveness  of  the  experimental  design,  data  from  eight 
of  these  pilots  (four  airline  and  four  DAC)  was  used  in  the  objective 
analysis  while  all  nine  pilots  were  represented  in  the  debriefing 
questionnaire  analysis. 

Facility 

Primary  experimental  investigation  took  place  in  the  DAC  Motion  Base 
Simulator  (MBS).  The  MBS  employs  a  six  axis  motion  system  and  a  high 
fidelity  terrain  model  for  external  visual  reference.  Since  motion  cues 
were  not  required  for  this  study,  the  simulator  was  operated  in  a  fixed 
base  mode.  Depending  on  the  test  conditions,  the  simulator  was  config¬ 
ured  with  either  EMADS  or  conventional  instruments.  Each  of  these 
configurations  can  be  seen  in  Figure  1  and  Figure  2.  Other  than  the 
engine  instrument  configuration  and  the  EMADS  keyboard  mounted  on  the 
forward  pedestal,  the  cockpit  layout  was  the  same  for  all  test  conditions. 
Active  displays  and  controls  used  in  the  study  included  the  following: 

1.  Control  wheel  and  column 

2.  Trim  switches 

3.  Fault  detection  switch  (mounted  on  control  wheel) 

4.  ADI,  HSD,  altimeter,  VSI,  IAS 

5.  Vertical  speed  wheel  and  control  switch 

6.  APU  isolation  value 

7.  Overide/airstart  switch 

8.  Landing  gear  lever 

9.  Flap/slat  controls 

10.  Stabilizer  position  indicator 

11.  Parking  brake 

12.  Master  caution  light 
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FIGURE  1.  CONVENTIONAL  INSTRUMENTATION 


13.  Overhead  fault  annunciator  and  correction  panel 

14.  EMADS  control  panel 

15.  Engine  instruments 

16.  .Fuel  levers,  start  switches,  and  throttles 

The  simulator  cockpit  was  designed  to  represent  that  of  a  three  engine 
wide  body  aircraft.  The  general  control /display  arrangement  was  similar 
to  that  of  a  DC-10,  as  were  the  aircraft  handling  qualities.  Pilots 
were  told  in  the  pretest  briefing,  however,  that  these  similarities  were 
coincidental . 

In  an  effort  to  measure  the  attention-getting  value  of  the  two  display 
types,  the  auto-pilot  disconnect  switch  was  made  to  function  as  a  fault 
detection  device.  Pilots  were  Instructed  to  depress  this  switch 
immediately  after  they  detected  the  existence  of  an  abnormal  condition. 
Other  than  this  modification,  all  displays  and  controls  were  representa¬ 
tive  of  standard  flight  deck  equipment. 

For  this  study,  the  simulator  was  configured  to  accommodate  a  pilot,  a 
first  officer,  and  two  experimenters.  A  special  control  panel  and 
multichannel  conmunl cation  system  allowed  one  of  the  experimenters  to 
exercise  control  over  the  test  environment  while  the  other  experimenter 
interacted  with  the  pilot  (test  subject)  and  co-pilot.  Communication 
headsets  were  provided  for  both  pilots  and  experimenters.  Each  was  tied 
into  an  audio-cassette  taping  system  designed  to  record  intracockpit 
conversation  for  subsequent  analysis.  The  experimenters  coumuni cation 
systems  were  tied  Into  three  additional  remote  locations  where  support 
personnel  were  stationed  to  receive  and  transmit  pertinent  information. 

Flight  Scenario 

To  make  a  discussion  of  the  experimental  design  more  meaningful,  a 
description  of  the  flight  scenario  used  for  each  test  run  Is  in  order. 
The  basic  flight  profile  can  be  seen  in  Table  1.  The  flight  plan 
consisted  of  a  modified  noise  abatement  takeoff  and  climbout  to  5,000 
feet.  A  pilot  from  the  Douglas  Aircraft  Flight  Operations  group 
participated  as  a  co-pilot,  and  assisted  In  such  tasks  as  gear 
retraction,  V-speed  callout  and  flap/slat  retraction.  The  co-pilot  did 
not  assist  in  the  fault  correction  or  flight  tasks. 
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TABLE  1 

FLIGHT  PROFILE  FOR  EMADS  -  CONVENTIONAL  INSTRUMENTS 
COMPARATIVE  ANALYSIS 

TAKEOFF  MAINTAIN  RUNWAY  HDG  UNTIL  2,000  FEET  (HOG  000°) 

REDUCE  TO  CLIMBTHRUST  AT  2,000  FEET  (103.3%  N-, ) 

MAINTAIN  V2  +  10  UNTIL  3,000  FEET 

AT  3,000  FEET  REDUCE  VERTICAL  SPEED  TO  1 ,000  FEET/MINUTE 
CONTINUE  AT  CLIMBTHRUST  UNTIL  CLEANUP  IS  COMPLETE 
FLAP  RETRACT  (181  KTS) 

SLAT  RETRACT  (299  KTS) 

CONTINUE  AT  CLIMBTHRUST  TO  5,000  FEET 


After  the  engines  were  started  and  set  at  ground  idle,  the  aircraft  was 
positioned  at  the  foot  of  the  simulated  runway.  After  takeoff,  at  an 
altitude  of  2,000  feet,  the  pilot  was  instructed  to  reduce  from  takeoff 
to  climb  thrust.  The  co-pilot  advised  the  pilot  when  the  simulator  was 
approaching  2,000  feet  so  that  the  thrust  value  could  be  adjusted.  The 
co-pilot  also  advised  the  pilot  when  the  simulator  neared  3,000  feet  and, 
on  command,  reduced  the  vertical  speed  to  the  appropriate  level.  In 
addition,  the  co-pilot  retracted  the  flaps  and  slats  on  the  pilots 
command. 

Experimental  Design 

The  study  was  designed  to  assess  a  wide  range  of  EMADS  operating 
characteristics.  Therefore,  a  number  of  subexperiments  were  conducted 
serially,  with  each  phase  of  activity  during  a  particular  test  trial 
contributing  data  to  one  of  five  separate  experiments.  Each  experiment 
was  designed  to  comparatively  analyze  EMADS  and  conventional  instruments 
relative  to  the  following  scheme: 

1.  Preflight  -  Normal  operations 

2.  Preflight  -  Fault  monitoring 

3.  Takeoff  -  Normal  operations 

4.  Takeoff  -  Fault  monitoring 

5.  Inflight  -  Fault  monitoring 

Each  subexperiment  will  first  be  described  separetely;  this  will  be 
followed  by  a  description  of  the  procedure  used  to  meet  the  objectives 
of  each  experiment  by  conducting  sixteen  test  trials.  Half  of  the 
trials  Included  engine  startup  and  all  involved  the  flight  profile 
shown  in  Table  1. 

Pref light:  Normal  Operations 

This  experiment  was  designed  to  assess  the  rapidity  with  which  the 
engines  could  be  started  using  EMADS  and  conventional  instruments. 

Figure  3  illustrates  the  experimental  design,  where  each  pilot  started 
all  engines  twice  with  both  EMADS  and  conventional  instruments.  No 
faults  were  introduced  during  these  trials  so  that  a  precise  measurement 
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of  total  start  time  could  be  obtained.  Start  time  was  defined  as  the 
interval  between  the  depression  of  the  start  switch  for  the  first 
engine  and  the  point  at  which  the  third  engine  was  at  ground  idle 
(25%  N-|).  These  four  trials  were  interspersed  among  trials  where 
faults  were  introduced  during  the  engine-start  sequence. 

Preflight:  Fault  Monitoring 

The  purpose  of  this  experiment  was  to  assess  the  relative  effectiveness 
with  which  pilots  could  detect  and  correct  power  plant  problems  during 
the  start  sequence  using  EMADS  and  Conventional  Instruments.  Figure  4 
represents  a  complete  factorial  design  where  each  pilot  was  exposed  to 
all  treatment  conditions. 

Of  the  sixteen  test  trials,  eight  involved  the  complete  start  sequence 
for  the  three  engines  while,  in  the  remainder,  the  simulator  was  set  at 
the  foot  of  the  runway  (with  all  engines  at  ground  idle),  ready  for 
takeoff. 

During  the  engine  start  procedure  for  a  particular  trial  where  EMADS  or 
conventional  instruments  were  used,  either  a  hot  or  hung  start  was 
introduced  in  one  of  the  engines.  The  definitions  and  appropriate 
corrective  actions  for  these  faults  as  well  as  those  introduced  during 
other  flight  phases  are  provided  in  Table  2.  During  the  engine  start 
segment  of  each  trial,  a  maximum  of  one  and  a  minimum  of  zero  faults 
were  Introduced. 

Takeoff  -  Normal  Operations 

This  experiment  was  designed  to  measure  the  efficiency  and  accuracy  with 
which  the  pilots  operated  the  engines  during  the  takeoff  roll.  As 
illustrated  In  Figure  5,  a  five  way  design  was  employed  with  two  levels 
of  each  independent  variable.  Repeated  measures  were  obtained  on  four 
of  these  variables  with  the  runway  visual  range  (RVR)  serving  as  a 
grouping  variable;  each  pilot  was  exposed  to  all  levels  of  the  repeated 
measures  variables  while  they  were  exposed  to  only  one  level  of  the 
grouping  variable.  In  this  way,  four  of  the  pilots  were  faced  with  a 
700  foot  RVR  while  the  remainder  were  provided  with  a  visual  range  of 
2,400  feet. 
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EVALUATION  CRITERIA  FAULT 


TABLE  2 


FAULT  CORRECTION  PROCEDURE 


FAULT 

DEFINITION 

CORRECTIVE  ACTION 

A.  HUNG  START 

EGT  <  200°  AFTER  25  SEC 

DETECT,  FUEL  LEVER  OFF, 
AFTER  30  SEC,  PULL  ST. 
SWITCH,  START  ENGINE 

B.  HOT  START 

EGT  >  750°  FOR  40  SEC 

OR  EGT  >  250°/  SEC  AT 

20  SEC  AFTER  FUEL 

LEVER  ON 

DETECT,  FUEL  LEVER  OFF, 
MOTOR  30  SEC,  PULL  ST. 
SWITCH,  START  ENGINE 

C.  Ni  OR  EGT 
EXCEEDANCE 

N,  116%  Ni 

ECT  >  875° 

DETECT,  PULL  THROTTLE 
QUICKLY,  BACK  ALMOST  TO 
IDLE  AND  THEN  BACK  TO 
NORMAL  POSITION 

D.  FLAMEOUT 

ONE  Mi  >  11%  LESS  THAN 

OTHER  N-i  WITH  AT  LEAST 

ONE  N]  AT  70% 

DETECT,  PUSH  OVERRIDE/ 
AIRSTART  SWITCH  TO  ON 
POSITION 

E.  AUXILIARY 
FAULTS 

MASTER  CAUTION  ON  AND 
OVERHEAD  ANNUNCIATOR  ON 

DETECT,  DEPRESS  OVERHEAD 
ANNUNCIATOR  SWITCH 

1.  SELECT  FLAP  LIMIT  OVRD 

2.  UPPER  YAW  DAMP  INOP 

3.  PNEU  TEMP  HIGH 

4.  R  WINDSHIELD  ANTI-ICE 

INOP 

-  mpm  v  wr*  .*  •’  */V.s . 


Depending  on  the  RVR  employed,  it  was  expected  that  pilots  would  spend 
more  or  less  time  in  the  "heads-up"  position.  It  was  further  hypothesized 
that  pilots  would  perform  better  (in  terms  of  power  setting  accuracy)  in 
the  head-up  position  using  EMADS  because  the  display  provides  a  positive 
indication  for  out-of- tolerance  conditions. 

All  pilots  participated  in  eight  trials  with  EMADS  and  eight  with  conven¬ 
tional  instruments.  In  addition,  half  the  trials  for  each  pilot  were 
initiated  with  a  static  takeoff  where  the  power  was  set  before  the 
parking  brake  was  released.  The  remainder  were  started  with  a  rolling 
takeoff  where  the  parking  brake  was  released  and  the  power  was  set 
during  the  takeoff  roll. 

An  interactive  effect  was  expected  where  better  power  setting  performance 
would  result  from  use  of  EMADS  during  a  rolling  takeoff  while  no 
difference  between  the  two  systems  would  be  found  during  static  takeoffs. 
This  expectation  was  born  out  of  the  fact  that  EMADS  provides  a  discrete 
cue  when  the  N-j  thrust  value  is  within  one  percent  of  the  command  para¬ 
meter.  During  rolling  takeoffs,  the  pilots  were  forced  to  divide  their 
time  between  control  of  the  aircraft  and  accurate  power  setting.  The  N-j 
target  value  was  either  high  (111.0%  N-j)  or  low  (108.3%  N-j )  for  each 
trial.  The  high  value  represented  normal  takeoff  thrust  while  the  lower 
value  represented  the  power  requirements  for  a  derated  takeoff.  It  was 
expected  that  pilots  would  tend  to  undershoot  the  high  command  value 
more  with  conventional  than  with  EMADS.  Without  the  direct  cue  provided 
with  EMADS,  It  was  felt  that  pilots  would  be  somewhat  conservative  to 
avoid  overboosting  the  engines.  Finally,  each  pilot  participated  in  two 
repetitions  of  each  experimental  condition. 

Standard  DC-10  operating  procedures  specify  that  takeoff  power  should  be 
set  by  the  time  the  aircraft  reaches  80  knots.  To  measure  the  relative 
ease  with  which  takeoff  power  could  be  set  with  EMADS  and  conventional 
Instruments,  at  80  knots  the  value  for  each  engine  was  recorded  in 
system  software  for  subsequent  analysis.  This  provided  an  index  of  the 
pilots  ability  to  accurately  set  the  power  while  maintaining  safe 
control  of  the  aircraft  with  each  engine  display  system.  Also,  by 
recording  the  mean  setting  at  for  each  pilot,  it  was  possible  to 


determine  how  much  throttle  movement  occurred  after  80  knots  with  each 
display  system;  the  difference  between  the  values  at  80  knots  and 
representing  the  degree  to  which  the  power  was  adjusted  after  80  knots. 

As  fuel  costs  rise,  so  too  does  the  need  to  judiciously  monitor  consumption 
during  all  phases  of  flight.  Since  a  significant  amount  of  fuel  is 
required  to  allow  the  aircraft  to  become  airborne,  it  was  decided  that 
fuel  consumption  per  hour  and  total  fuel  usage  would  be  monitored  for 
each  treatment  condition  during  the  takeoff  roll.  These  measurements 
were  used  as  indices  of  fuel  savings  as  a  function  of  display  type,  with 
the  remaining  treatment  conditions  providing  interactive  effects. 

Takeoff  -  Fault  Monitoring 

This  experiment  was  designed  to  measure  the  degree  to  which  pilots  could 
detect  and  correct  failures  while  safely  controlling  the  aircraft  after 
rotation.  As  with  the  previous  experiment,  RVR  was  employed  as  a 
grouping  variable  with  repeated  measures  on  the  three  remaining  variables 
(Figure  6).  For  eight  of  the  sixteen  trials,  an  or  EGT  exceedance  was 
introduced  somewhere  between  ten  and  nineteen  seconds  after  rotation. 

The  definitions  and  corrective  actions  associated  with  these  faults  can 
be  seen  in  Table  2.  Pilots  were  instructed  to  correct  the  problem  while 
maintaining  safe  control  of  the  aircraft.  For  a  period  of  ten  seconds 
beginning  at  the  onset  of  the  fault,  heading  error  (in  degrees)  was  recorded 
at  a.  rate  of  twenty  samples  per. second.  This  provided  a  measure  of  the' 
pilots'  ability1  to.  maintain  control  of . the  aircraft. 'while  engaged  In 
fault  correction  activities.  The  time  to  detect  and  correct  each  fault 
was  also  recorded  as  was  the  degree  or  percent  of  exceedance.  These 
represented  measures  of  potential  engine  damage  caused  by  failure  to 
correct  the  faults  in  an  expedient  manner. 

Inflight  -  Fault  Monitoring 

The  last  experiment  was  similar  to  that  involving  fault  monitoring  after 
rotation.  The  primary  differences  centered  around  the  faults  and  their 
timing,  as  Illustrated  In  Figure  7.  As  with  the  previous  experiment,  the 
pilots  were  divided  into  two  groups  of  four,  being  exposed  either  to  a 


18 


700  or  2,400  foot  RVR.  Again,  half  the  trials  for  each  pilot  were  run 
with  EMADS  and  the  remainder  with  conventional  instruments.  Also,  pilots 
were  exposed  to  two  repetitions  of  each  fault;  during  eight  of  the 
sixteen  test  trials,  either  an  engine  flameout  or  EOT  exceedance  was 
Introduced  at  some  point  between  five  and  twenty  seconds  after  the 
simulator  reached  3,000  feet.  The  definitions  and  corrective  procedures 
for  these  faults  are  contained  in  Table  2.  The  evaluation  criteria  were 
the  same  as  those  used  in  the  previous  experiment  (Takeoff:  Fault 
Monitoring). 


Auxiliary  Fault  Monitoring 

In  addition  to  the  engine  faults  that  were  introduced  during  each  trial, 
a  number  of  non-engine-related  faults  were  presented  at  various  times 
during  the  flight.  These  faults  were  annunciated  by  a  master  caution 
light  and  on  an  annunciator  matrix  which  was  located  on  the  forward 
overhead  panel.  These  conditions  were  introduced  to  expand  the  pilot's 
eye  reference  scan  to  Include  other  areas  of  the  cockpit  typically 
viewed  during  abnormal  operations.  The  faults  employed  Included  the 
following: 

1.  Select  flap  limit  override 

2.  Upper  yaw  damper  Inoperative 

3.  Pneumatic  temperature  high 

4.  Right  windshield  anti-ice  Inoperative 

The  corrective  action  required  for  each  of  these  faults  is  shown  In 
Table  2. 

Trial  Assignments 

In  an  effort  to  consolidate  the  activities  required  to  obtain  the 
necessary  data  for  each  experiment,  a  series  of  sixteen  test  trials  or 
"cells"  were  designed  such  that  all  experimental  objectives  could  be  met 
in  a  reasonable  length  of  time.  As  can  be  seen  In  Table  3,  each  cell 
represents  a  unique  combination  of  variables  for  each  experiment.  Table 
4  provides  an  explanation  of  the  information  corresponding  to  each 
flight  phase  for  Cell  2. 
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DISTRIBUTION  AND  TIMING  FOR  EACH  TRIAL 
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Cells  5  through  8  and  13  through  16  represent  trials  where  the  simulator 
was  reset  at  the  foot  of  the  runway  with  all  engines  at  ground  idle.  No 
engine  startup  was  required  for  these  trials  because  all  necessary  data 
for  the  first  two  experiments  was  collected  during  the  eight  trials 
shown  in  Table  3  (Cells  1  through  4  and  9  through  12).  During  trials 
where  faults  were  introduced,  the  engines  where  the  failures  occurred 
were  selected  at  random.  To  avoid  any  systematic  learning  effects  half 
the  pilots  completed  Cells  1  through  8  first,  followed  by  Cells  9 
through  16.  The  remainder  completed  the  sixteen  trials  in  reverse  order. 
A  Latin  Square  design  was  used  to  counterbalance  the  order  in  which  the 
cells  were  completed  by  each  pilot  for  both  EMADS  and  conventional 
instruments,  thus  allowing  each  pilot  to  complete  the  sixteen  trials  in 
a  unique  order  (Appendix  A). 

Table  3  represents  the  actual  trial  assignment  sheet  used  during  the 
study.  A  separate  assignment  sheet  was  used  for  each  pilot  so  that  the 
run  number  corresponding  to  each  cell  and  the  RVR  could  be  appropriately 
adjusted.  As  mentioned  earlier,  the  counterbalancing  scheme  was  such 
that  all  pilots  would  complete  each  cell  as  a  different  test  run.  Also, 
since  the  visibility  conditions  (700  or  2,400  foot  RVR)  were  varied, 
space  was  allotted  for  this  entry  as  well. 

Procedure 

A  standard  set  of  test  procedures  was  developed  to  serve  as  a  means  to 
ensure  continuity  across  pilots  relative  to  information  transfer  and 
activity  control  during  the  study.  A  full  day  of  testing  lasted 
approximately  six  hours.  The  following  are  the  activities  that  were 
carried  out  during  each  test  session  along  with  their  timing.  Each  of 
these  activities  will  be  described  briefly  in  the  following  pages  with 
a  more  detailed  description  provided  in  Appendix  B. 


Activity 


1.  Preflight  briefing 

2.  Cockpit  briefing 

3.  Practice  trials 

4.  Test  trials 


Cumulative  Hours 
0000  -  0030 
0030  -  0130 
0130  -  0200 
0200  -  0245 
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5. 

System  changeover 

0245  -  0415 

6. 

Briefing  for  second  configuration 
and  practice  trials 

0415  -  0500 

7. 

Test  trials  for  second  configuration 

0500  -  0545 

8. 

Debriefing 

0545  -  0615 

Pref light  Briefing 

In  most  cases,  the  study  session  began  at  approximately  8:30  a.m.  In 
one  case,  because  of  a  particular  travel  schedule,  the  session  began  at 
11:00  a.m.  and  concluded  at  approximately  5:00  p.m. 

All  pilots  invited  to  participate  were  provided  with  literature  that 
described  the  pertinent  details  of  the  study.  They  were  also  provided 
with  an  EMADS  operators  guide  and  an  article  that  described  earlier 
developmental  work  completed  on  the  EMADS  program.  The  first  part  of 
the  preflight  briefing  was  devoted  to  answering  questions  on  this 
material.  Following  this,  pilots  were  provided  with  a  description  of 
the  days  activities  and  the  facility  to  be  used.  On  days  when  EMADS 
was  to  be  tested  first,  a  videotape  of  a  number  of  EMADS  operating 
characteristics  was  presented.  This  lasted  approximately  twelve 
minutes,  after  which  a  brief  discussion  took  place  where  pilots  made 
comments  on  .and  asked  questions  about  EMADS.  On  days  where  EMADS  was 
tested  in  the  afternoon,  the  videotape  was  presented  during  the  system 
changeover  time.  In  either  case,  at  the  conclusion  of  the  videotape, 
the  cockpit  briefing  began  in  the  flight  simulator. 


Cockpit  Briefing 

This  period  of  time  was  structured  to  provide  the  pilot  with  hands-on 
familiarization  in  the  test  environment.  Each  of  the  following  items 
or  groups  of  Items  were  discussed,  a  more  detailed  description  of  which 
can  be  found  In  Appendix  B. 

1.  General  cockpit  configuration 

2.  Voice  taping  system 

3.  Active  displays  and  controls 

4.  Simulator  Idlosyncracles 

5.  Visual  scene 
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6.  Flight  plan 

7.  Co-pilot  activities 

Practice  Trials 

Prior  to  "flying"  the  simulator,  pilots  were  familiarized  with  the  engine 
Instruments  (EMADS  or  conventional)  and  given  several  opportunities  to 
set  takeoff  power  in  the  static  configuration.  This  was  done  to  provide 
a  "feel"  for  the  throttle  dynamics  relative  to  the  information  displayed 
on  the  Instruments.  At  this  point,  pilots  were  given  the  opportunity  to 
become  familiar  with  the  handling  qualities  of  the  simulator  by  excer- 
cising  the  flight  plan.  During  these  trials,  no  faults  were  introduced 
as  the  main  objective  was  to  master  the  flight  task. 

After  the  pilot  was  confortable  with  the  flight  scenario,  definitions 
and  illustrations  were  provided  for  all  faults  and  the  action  required 
to  correct  them.  At  this  point,  a  "walk  through”  was  conducted  where  a 
number  of  test  scenarios  were  completed  such  that  the  pilot  would  be 
exposed  to  all  failure  conditions  that  were  to  occur  during  the  test 
session.  During  these  trials,  the  pilot  was  cued  by  the  experimenter 
just  prior  to  the  occurrence  of  each  failure  condition.  This  allowed 
the  pilot  to  see  exactly  how  the  displays  would  appear  as  faults 
situations  were  developing.  Cues  were  also  provided,  as  necessary,  for 
the  corrective  action.  This  entire  sequence  (presentation  of  all  faults) 
was  repeated  with  the  experimenter  providing  no  cues.  This  standardized 
practice  scenario  was  designed  to  reduce  the  "warm-up"  effects  that 
generally  accompany  test  designs  that  require  subjects  to  perform  in 
situations  with  which  they  are  not  familiar. 

Test  Trials 

Depending  on  the  engine  display  configuration  Installed  for  this 
portion  of  the  test  session,  the  pilot  completed  the  first  eight  trials 
with  either  EMADS  or  conventional  Instruments.  During  this  time,  half 
of  the  required  data  for  one  pilot  was  collected. 
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System  Changeover 

After  completing  the  test  trials  with  the  first  display  configuration, 
approximately  ninety  minutes  were  required  to  install  the  engine 
instruments  for  the  afternoon  test  session.  Part  of  this  time  was  used 
for  a  lunch  break  while  the  remainder  was  used  to  begin  the  briefing  for 
the  afternoon  activities.  On  days  when  EMADS  was  scheduled  for  the 
afternoon,  part  of  this  time  was  used  to  present  the  videotape  described 
earlier,  and  to  discuss  the  various  EMADS  operating  modes. 

Briefing  for  Second  Display  Configuration  and  Practice  Trials 
Because  the  new  engine  display  configuration  represented  the  only  change 
in  cockpit  hardware,  a  majority  of  the  briefing  time  was  used  to 
familiarize  the  pilot  with  this  aspect  of  the  test  environment. 

Following  this,  the  s ame  series  of  familiarization  and  parctice  trials 
used  for  the  morning  session  was  employed. 

Test  Trials 

The  eight  trials  required  to  obtain  the  remaining  data  were  completed  at 
this  time.  Except  for  the  engine  display  configuration,  all  test  para¬ 
meters  were  the  same  as  those  used  in  the  morning  session. 

Debriefing 

After  the  final  test  trial,  pilots  were  Invited  to  a  debriefing  session 
held  at  a  location  outside  the  simulation  facility.  During  this  time, 
pilots  were  asked  to  complete  a  debriefing  questionnaire  which  solicited 
their  views  on  the  relative  merits  of  EMADS  and  conventional  Instruments 
with  respect  to  a  number  of  operational  criteria.  A  copy  of  the 
debriefing  questionnaire  can  be  found  in  Appendix  C.  In  addition,  an 
informal  discussion  took  place  where  relevant  pilot  comments  were 
recorded  for  subsequent  documentation. 
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RESULTS 


Statistical  Analysis  Technique 

An  analysis  of  variance  (BMDP2V)  was  performed  on  each  of  the  following 
sets  of  data: 

1.  Preflight:  Normal  operations 

2.  Preflight:  Fault  monitoring 

3.  Takeoff:  Normal  operations 

4.  Takeoff:  Fault  monitoring 

5.  Takeoff:  Degree  of  EGT  and  N-j  exceedance 

6.  Inflight:  Fault  monitoring 

7.  Inflight:  Degree  of  EGT  exceedance 

8.  Takeoff  and  inflight:  Heading  deviation 

The  results  of  each  of  these  analyses  are  discussed  in  the  following 
pages.  Summary  tables  for  each  analysis  can  be  found  In  Appendix  D. 

Preflight  -  Normal  Operations 

Eight  of  the  sixteen  test  trials  required  the  pilot  to  start  each  of  the 
three  engines  using  standard  DC-10  procedures.  For  the  remaining  trials, 
the  simulation  was  Initiated  at  the  foot  of  the  runway  with  all  engines 
at  ground  idle,  ready  for  takeoff.  During  four  of  the  engine  startup 
trials  (two  each  with  EMADS  and  conventional  instruments)  no  faults  were 
introduced  and  the  time  required  to  start  all  engines  was  recorded. 

This  time  frame  was  initiated  when  the  start  switch  for  the  first  engine 
was  depressed  and  ended  when  the  third  engine  was  at  ground  idle.  This 
time  segment  was  of  interest  because  in  some  strategic  missions  such  as 
military  airlifts,  the  time  required  to  prepare  the  aircraft  for  takeoff 
is  of  critical  importance  to  operational  success. 

A  two  way  analysis  of  variance  with  repeated  measures  was  performed  on 
the  results,  which  exposed  a  significant  reduction  in  total  startup 
time  when  conventional  Instruments  were  used  (p<.05).  Figure  8 
illustrates  these  results.  This  difference  was  partly  due  to  the  fact 
that  the  pilots  were  using  slightly  different  cues  with  the  two  systems. 
EMADS  provided  a  cue  to  start  the  next  engine  when  the  engine  being 


attended  to  reached  twenty-five  percent  (25%)  of  its  maximum  N]  thrust 
value.  Shortly  after  receiving  this  cue,  pilots  would  initiate  the 
start  procedure  for  the  next  engine.  Although  instructed  that  twenty- 
five  percent  (25%)  N-j  represented  normal  ground  idle  with  conventional 
instruments  as  well,  a  majority  of  the  pilots  also  used  cues  provided 
by  the  digital  readout  of  exhaust  gas  temperature  (EGT)  on  the  conven¬ 
tional  display  while,  for  the  most  part,  they  did  not  make  use  of  this 
parameter  on  the  EMADS  display.  Normal  DC-10  procedures  specify  that 
an  engine  is  considered  as  being  at  ground  idle  when  the  EGT  reaches 
six  hundred  and  two  degrees  Fahrenheit  (602°F.),  which  corresponds  to  an 
N.j  value  of  approximately  twenty-one  and  eight  tenths  percent  (21.8%). 
Although  the  simulation  cockpit  was  not  meant  to  represent  that  of  a 
DC-10,  it  is  understandable  that  the  pilots  would  see  the  similarities 
and,  at  times,  revert  to  procedural  activities  associated  with  that 
aircraft.  This  discrepancy  was  responsible  for  some,  but  not  all  of  the 
difference.  Inspection  of  the  raw  data  indicates  that  four  of  the  eight 
pilots  took  an  abnormally  long  time  to  start  the  engines  on  their  first 
repetition  with  EMADS  (forty  to  seventy  seconds  above  the  mean).  Such 
differences  did  not  occur  during  the  second  repetition  and  thus,  very 
little  difference  exists  between  EMADS  and  conventional  instruments. 

The  large  discrepancy  for  these  four  pilots  may  have  been  due  to 
learning  effects. 

Because  EMADS  is  capable  of  monitoring  all  engine  parameters  during  each 
flight  phase  (including  startup),  it  should  be  feasible  to  provide  cues 
based  on  EGT,  N.|,or  any  combination  of  engine  parameters  that  will 
expedite  engine  starting. 


Preflight;  Fault  Monitoring 

During  four  trials  (two  each  for  EMADS  and  conventional  instruments), 
either  a  hot  or  a  hung  start  was  introduced  in  one  of  the  engines  during 
the  start  process.  Pilots  were  instructed  to  take  appropriate  corrective 
action  as  soon  as  the  problem  became  evident. 


Approximately  seventeen  seconds  elapsed  from  the  time  the  faults  were 
initiated  in  the  system  software  until  rate  changes  were  sensed  by  both 
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engine  displays.  All  data  were  transformed  (seventeen  second  subtraction) 
to  yield  a  more  representative  set  of  detection  and  response  times.  A 
two  way  analysis  of  variance  with  repeated  measures  was  performed  on  the 
results  which  are  illustrated  in  Figures  9  and  10.  In  terms  of  detection 
and  response  time,  performance  was  better  in  correcting  the  hot  start 
condition  when  EMADS  was  used  while  conventional  instruments  produced  the 
best  performance  on  the  hung  start  condition.  The  response  time  data 
provide  a  good  illustration  of  the  system  monitoring  processes  used  by 
the  pilots  and  by  EMADS.  For  this  study,  a  hung  start  was  defined  as  a 
twenty-five  second  interval  after  the  fuel  flow  was  initiated  where  the 
E6T  remained  at  or  below  two  hundred  degrees  Fahrenheit  (200°  F.).  A 
hot  start,  on  the  other  hand,  was  when  the  E6T  was  greater  than 
seven  hundred  and  fifty  degrees  Fahrenheit  (750°F.)  for  forty  seconds 
or  the  change  in  the  EGT  was  greater  than  twenty-five  degrees  per 
second  at  twenty  seconds  after  the  fuel  flow  was  initiated.  Obviously, 
a  hung  start  was  much  easier  to  Identify  than  a  hot  start  because  it 
represented  a  discrete  event  where  no  rate-of-change  calculations  were 
required.  In  fact,  the  pilots  actually  performed  better  with  conventional 
instruments  because  they  realized  a  hung  start  was  imminent  slightly 
before  the  twenty-five  seconds  called  out  in  the  fault  definition  while, 
with  EMADS,  most  pilots  waited  for  the  cue  on  the  display  screen  which 
was  not  presented  until  twenty-five  seconds  after  fuel  flow  initiation. 
This  represents  an  area  where  software  modifications  could  provide  a 
more  expedient  cue  when  this  fault  occurs.  The  response  time  data  for 
the  hot  start  provides  an  illustration  of  the  primary  strength  of  EMADS: 
the  ability  to  monitor  rate  changes  and  provide  status  information  on  a 
real-time  basis.  This  is  evidenced  by  the  improved  performance  resulting 
from  the  use  of  EMADS  during  the  hot  start  condition.  It  should  also  be 
noted  that  the  performance  differences  between  the  two  display  systems 
were  not  statistically  significant. 

Takeoff:  Normal  Operations 


1.  Mean  N-j  setting  at  80  knots 

2.  Mean  N-j  setting  at 

3.  Throttle  adjustment  between  80  knots  and 

At  80  knots  and  at  ,  the  N-j  value  was  measured  and  recorded  as  a  means 
to  evaluate  throttle  setting  accuracy.  Standard  DC- 10  operating  proce¬ 
dures  state  that  takeoff  power  should  be  set  by  80  knots  and  that  both 
of  the  pilots  hands  should  be  on  the  control  wheel  by  (the  velocity 
after  which  the  takeoff  cannot  be  safely  aborted). 

An  analysis  of  variance  was  performed  on  both  sets  of  data,  the  results 
of  which  are  illustrated  in  Figures  11 A  and  11B.  As  can  be  seen  in 
these  figures,  the  pilots  consistently  set  power  more  accurately  with 
EMADS,  although  these  differences  in  accuracy  were  not  statistically 
significant.  With  conventional  instruments,  the  pilots  tended  to  under¬ 
boost  the  engines  more  so  than  with  EMADS.  This  may  have  been  due  to 
the  fact  that  a  positive  indication  of  within-limit  power  setting  was 
provided  on  the  EMADS  display.  It  may  be  that  the  absence  of  such  an 
indication  on  the  conventional  dials  caused  the  pilots  to  exercise 
additional  caution  to  avoid  overboosting  the  engines. 

An  analysis  of  variance  was  also  performed  on  the  difference  between  the 
N-j  value  at  V1  and  80  knots.  This  provided  a  measure  of  the  amount  of 
throttle  adjustment  being  made  after  80  knots.  As  can  be  seen  in 
Figure  12,  more  throttle  adjustment  was  made  after  80  knots  with  EMADS 
than  conventional  instruments.  This  difference  however,  was  not 
statistically  significant. 


Takeoff:  Fault  Monitorinc 


Shortly  after  takeoff  for  eight  of  the  sixteen  trials  (four  each  for 


EMADS  and  conventional  instruments),  either  an  or  EGT  exceedance  was 
presented.  Pilots  were  instructed  to  correct  the  problem  quickly  and 


accurately  while  maintaining  safe  and  accurate  control  of  the  aircraft. 


An  analysis  of  variance  was  performed  on  the  results  which  are 
illustrated  in  Figures  13  and  14.  In  terms  of  detection  and  response 
times,  performance  was  significantly  better  when  EMADS  was  employed 
(P<.05).  This  was  primarily  due  to  the  fact  that,  with  EMADS,  a 
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FIGURE  11A.  TAKEOFF:  NORMAL  OPERATIONS 
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FIGURE  11B.  TAKEOFF:  NORMAL  OPERATIONS 
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positive  indication  (flashing  engine  legend)  was  provided  for  both  faults 
while,  with  conventional  instruments,  a  positive  indication  was  provided 
only  for  the  EGT  exceedance.  The  indication  provided  by  EMADS  was 
significantly  more  obtrusive  than  that  provided  by  conventional 
instruments.  Given  the  fact  that  both  displays  were  located  outside  the 
pilots  primary  field  of  vision,  this  capability  provided  a  distinct 
advantage  to  EMADS  which  is  illustrated  in  Figures  13  and  14.  It  is 
likely  that  these  performance  differences  would  shrink  significantly  if 
both  engine  monitoring  systems  were  tied  into  the  master  caution  system. 

An  analysis  was  also  performed  on  the  degree  (measured  in  percent)  of 
and  EGT  exceedance  that  occured  during  the  fault  correction  process. 

This  measurement  represented  the  value  of  the  or  EGT  parameter  at  the 
point  where  corrective  action  was  initiated,  less  the  normal  value. 

When  viewed  along  with  the  response  time  measurements,  a  clear  picture  of 
potential  engine  damage  emerges  in  terms  of  both  degree  and  duration  of 
the  exceedance. 

Analyses  of  variance  were  performed  on  the  N1  and  EGT  exceedance  values 
for  both  EMADS  and  conventional  instruments,  the  results  of  which  are 
illustrated  in  Figure  15.  The  use  of  EMADS  resulted  in  a  statistically 
significant  reduction  in  exceedance  values  for  the  parameter  (p< .01 ) . 
Although  a  lower  level  of  exceedance  was  also  found  with  the  EGT  faults 
when  EMADS  was  employed,  this  difference  was  not  statistically  signif¬ 
icant.  A  high  correlation  can  be  seen  when  a  comparison  is  made  between 
these  results  and  those  obtained  in  the  takeoff  fault  monitoring  analysis, 
where  response  time  differences  were  generally  more  pronounced  for  the 
N-j  exceedances. 


Inflight:  Fault  Monitoring 

During  eight  of  the  sixteen  trials  (four  each  for  EMADS  and  conventional 
instruments),  either  an  engine  flameout  or  EGT  exceedance  was  initiated 
at  a  predetermined  time  after  the  aircraft  passed  3,000  feet.  Again, 
pilots  were  instructed  to  take  corrective  action  while  maintaining  safe 


and  accurate  control  of  the  aircraft. 


FIGURE  15.  TAKEOFF:  FAULT  MONITORING 
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An  analysis  of  variance  performed  on  the  results  revealed  no  statistically 
significant  differences  in  terms  of  detection  and  response  times. 

Depending  on  the  fault  involved,  performance  was  slightly  better  with 
EMADS  or  conventional  instruments,  as  illustrated  in  Figures  16  and  17. 

The  relatively  low  workload  during  this  flight  segment  could  account  for 
the  lack  of  statistical  significance  in  the  results.  It  is  believed 
that  the  attention  getting  and  performance  benefits  associated  with 
EMADS  will  generally  increase  with  the  level  of  visual  task  loading. 

An  analysis  of  variance  was  also  performed  on  the  degree  of  EGT 
exceedance  and,  as  with  the  detection  and  response  time  data,  no 
statistically  significant  differences  were  found. 

Takeoff  and  Inflight:  Heading  Deviation 

During  each  trial,  heading  deviation  was  measured  and  recorded  for  ten 
second  Intervals  during  the  takeoff  and  inflight  segments.  When  faults 
were  introduced,  the  ten  second  intervals  were  initiated  concurrently 
with  the  onset  of  the  fault.  When  no  faults  were  introduced,  the  time 
interval  was  initiated  at  a  predetermined  time  that  corresponded  to  one 
of  the  fault  onset  times.  By  measuring  flight  task  performance  during 
periods  where  fault  detection  and  correction  was  occurring,  it  was 
possible  to  assess  the  distracting  effects  that  the  alerting  process 
imposed  on  the  flight  task. 

Due  to  a  software  computational  problem,  RMS  values  for  heading  devation 
were  not  available  for  analysis.  Instead,  an  analyses  of  variance  was 
performed  on  the  average  deviations  for  the  raw  data.  This  analysis 
revealed  no  statistically  significant  differences  between  EMADS  and 
conventional  instruments. 


Debriefing  Questionnaire 

Immediately  following  the  test  session,  each  pilot  was  asked  to  complete 
a  debriefing  questionnaire  which  addressed  the  relative  merits  of  EMADS 
and  conventional  Instruments.  The  results  of  this  questionnaire  are 
shown  In  Figure  18  and  Table  5.  Overall,  the  pilots  felt  that  EMADS 
was  superior  in  terms  of  system  monitoring,  fault  monitoring,  workload 
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Figure  18.  Questionnaire  Results  -  Mean  Pilot  Ratings 


TABLE  5 


QUESTIONNAIRE  RESULTS 


DO  YOU  FORESEE  ANY  PROBLEMS  IN  TRAINING  PILOTS  EXPERIENCED  WITH 
CONVENTIONAL  INSTRUMENTS  IN  THE  USE  OF  EMADS? 


YES  0 

NO  9 

NOT  SURE  0 

DO  YOU  FORESEE  ANY  PROBLEMS  IN  TRAINING  PILOTS  TO  USE  CONVENTIONAL 
INSTRUMENTS  IF  THEIR  ONLY  PREVIOUS  EXPERIENCE  IS  WITH  EMADS? 

YES  2 

NO  5 

NOT  SURE  1 


OVERALL,  WHICH  SYSTEM  WOULD  YOU  PREFER  IN  FUTURE  AIRCRAFT? 

9 
0 


EMADS 

CONVENTIONAL 
ANOTHER  TYPE  OF  DISPLAY 


0 


reduction,  and  power  setting  accuracy.  A  majority  could  foresee  no 
transfer  of  training  problems  for  pilots  transitioning  between  EMADS 
and  conventional  instruments.  Overall,  EMADS  was  the  clear  preference 
of  all  pilots  questioned.  An  additional  summary  of  the  questionnaire 
results  and  conments  can  be  found  in  Appendix  E. 

The  pilots  provided  a  number  of  useful  comments,  both  during  the  test 
session  and  as  part  of  the  debriefing  questionnaire.  A  summary  of  the 
comments  made  most  frequently  can  be  seen  in  Table  6.  In  general,  they 
felt  that  the  EMADS  display  was  easier  to  read  and  incorporate  into 
their  scan  pattern;  they  also  liked  the  obtrusive  attention  getting 
cues  and  the  automated  checklists.  A  number  of  the  pilots  stated  that 
the  effectiveness  of  EMADS  could  be  enhanced  with  the  application  of 
color. 


SUMMARY  OF  PILOT  COMMENTS 


COMMENTS 


EMADS  WAS  EASIER  TO  INCORPORATE  INTO  THE  SCAN  PATTERN 
AND  EASIER  TO  MONITOR  FOR  ABNORMALITIES 


EMADS  WAS  MORE  ATTENTION  DEMANDING  DURING  FAULT  CONDITIONS 
EMADS  WAS  EASIER  TO  READ 

EMADS  WAS  SIMPLIER  IN  TERMS  OF  DISPLAY  ORGANIZATION 
THE  USE  OF  COLOR  WOULD  INCREASE  THE  EFFECTIVENESS  OF  EMADS 
EMADS  PROVIDED  A  USEFUL  CHECKLIST  CAPABILITY 


NUMBER  OF 
PILOTS 


DISCUSSION 


A  number  of  efforts  have  been  undertaken  as  part  of  a  program  to  develop 
and  improve  the  EMADS  concept  involving  personnel  from  Douglas  Aircraft, 
General  Electric,  and  a  large  number  of  representatives  from  various 
airlines,  manufacturers,  and  regulatory  agencies.  The  major  thrust  of 
each  of  these  efforts  was  to  develop  and  subjectively  assess  the 
effectiveness  of  EMADS  relative  to  both  stated  design  objectives  and  the 
current  capability  provided  by  conventional  engine  instruments.  Each  of 
these  efforts  has  produced  results  that  reflect  quite  favorably  on  EMADS. 
After  obtaining  these  results,  it  was  determined  that  a  study  designed  to 
generate  objective  performance  data  should  be  conducted  to  validate  the 
EMADS  concept  relative  to  current  engine  control  and  monitoring 
capabilities. 

This  study  involved  a  series  of  experiments  designed  to  measure  pilot 
performance  under  a  variety  of  conditions  using  EMADS  and  conventional 
instruments.  It  was  hypothesized  that  EMADS  would  be  as  good  as  or 
better  than  conventional  instruments  in  terms  of  performance  on  the 
flight  and  fault  correction  tasks  for  each  experiment.  Each  experiment 
provided  a  unique  body  of  data  that  was  used  to  assess  the  relative 
utility  of  the  two  display  systems;  in  addition,  subjective  data  was 
collected  by  way  of  a  debriefing  questionnaire  as  well  as  through 
experimenter-pilot  interaction  during  the  test  session. 

For  a  majority  of  the  experiments,  the  data  validated  the  EMADS  concept 
in  terms  of  pilot  performance  when  compared  to  conventional  instruments. 

This  was  the  case  for  all  fault  monitoring  tasks  as  well  as  the  subjective 
input  provided  by  the  pilots.  The  only  exception  was  during  the  normal - 
operations  portion  of  the  preflight  activities  where  significantly 
less  time  was  required  to  start  all  engines  when  conventional  instruments 
were  used.  This  time,  reduction  was  only  evident  during  the  first  repetition 
while  no  significant  performance  differences  were  found  on  the  second 
Iteration.  This  suggests  that  Inadequate  training  with  EMADS  may  have  been 
the  source  of  this  difference.  It  was  also  noted  that  software  modifications 
to  EMADS  could  provide  more  expedient  action  cures  based  on  any  appropriate 
combination  of  engine  parameters.  Based  on  these  considerations,  the 
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performance  differences  described  in  the  results  section  do  not 
represent  an  area  of  special  concern. 

Analysis  of  the  preflight  fault  monitoring  data  revealed  comparable 
performance  levels  for  the  two  display  systems.  The  small  performance 
differences  that  were  found  did  not  consistently  favor  either  system. 
This  experiment  illuminated  one  of  the  potential  advantages  of  EMADS 
which  is  the  capability  to  automatically  monitor  system  abnormalities 
that  occur  as  a  function  of  parameter  rate  changes.  EMADS  resulted  in 
noticeably  better  performance  with  the  hot  start  than  with  the  hung 
start;  it  is  believed  that  this  difference  was  due  to  the  fact  that 
early  identification  of  a  hot  start  condition  requires  more  complex 
computational  activity.  High  speed  computers  can  carry  out  this  type 
of  activity  significantly  faster  and  more  accurately  than  humans  and 
thus,  a  potential  operational  advantage  of  EMADS  was  exposed. 

During  the  takeoff  roll,  no  significant  performance  differences  were 
found,  although  power  setting  was  slightly  more  accurate  when  EMADS  was 
used.  EMADS  provided  a  positive  indication  (brightened  chevron)  when 
the  N-|  value  was  within  ±  one  percent  of  the  command  value.  It  is 
believed  that  by  reducing  these  limits  to  -  one  half  of  one  percent,  a 
significant  improvement  in  power  setting  accuracy  could  be  realized  with 
EMADS. 

The  use  of  EMADS  resulted  In  significantly  better  performance  on  the 
fault  monitoring  task  that  occured  after  takeoff.  Performance  was 
better  in  terms  of  response  time,  detection  time,  and  percent  of 
exceedance  on  both  the  and  EGT  parameters.  The  obtrusive  indications 
provided  by  EMADS  for  abnormal  conditions  were  largely  responsible  for 
these  performance  differences. 


% 


% 


m 


Use  of  the  two  display  types  resulted  in  comparable  performance  levels 
for  fault  monitoring  during  the  inflight  segment.  No  significant 
performance  differences  were  found  in  the  measurements  of  response  time, 
detection  time,  or  percent  of  EGT  exceedance.  It  is  believed  that  the 
relatively  low  level  of  visual  task  loading  was  primarily  responsible 
for  the  lack  of  significant  results.  The  pilots  were  able  to  include 
the  engine  display  in  thier  scan  pattern  somewhat  more  frequently  than 
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would  normally  be  the  case.  Given  a  higher  level  of  visual  workload,  it 
is  likely  that  the  use  of  EMADS  would  result  in  measurably  better 
performance.  The  low  level  of  visual  workload  also  resulted  in  a 
minimal  performance  difference  between  the  two  display  types  in  terms  of 
heading  deviation.  The  lack  of  significance  in  the  fault  monitoring  and 
flight  task  performance  can  be  used  to  highlight  some  of  the  more 
significant  problems  encountered  in  attempting  to  conduct  a  sophisticated 
simulation  exercise.  One  of  the  main  drawbacks  of  simulating  abnormal 
operations  is  that  subjects  are  exposed  to  an  unusually  high  number  of 
failure  conditions  in  a  relatively  short  period  of  time.  This  tends  to 
encourage  or  produce  a  mental  "set"  where  subjects  devote  more  attention 
to  system  monitoring  than  would  be  the  case  in  an  actual  operational 
environment.  This  generally  results  in  smaller  performance  differences 
between  experimental  treatment  conditions.  In  addition  the  time 
constraints  associated  with  simulator  scheduling  do  not  allow  much  time 
for  test  design  revision  to  refine  the  experimental  paradigm.  In  one 
case,  last  minute  changes  were  required  because  of  simulator  malfunctions 
that  reduced  the  workload  level  during  the  inflight. segment  of  the  study. 
Because  it  is  impossible  to  anticipate  the  specific  effects  of  such 
changes,  an  assessment  of  the  potential  impact  must  be  made  on  a  case-by¬ 
case  basis  and,  in  most  instances,  tradeoffs  are  required  if  the  study  is 
to  continue.  Finally,  the  cost  required  to  conduct  a  full  mission 
simulation  is  generally  higher  than  that  which  can  reasonably  be  absorbed 
by  an  internal  research  and  development  budget.  Because  of  this,  some 
degree  of  realism  must  be  sacrificed  In  order  to  remain  within  the 
predetermined  financial  limits.  This  serves  to  reduce  the  face  validity 
of  the  simulation  environment  and  restrict  the  level  to  which  results  can 
be  generalized  to  the  "real  world". 

The  subjective  data  produced  results  slmiliar  to  those  of  previous  efforts. 
The  pilots  showed  a  clear  preference  for  EMADS  in  each  area  addressed  by 
the  questionnaire.  They  also  felt  that  transfer  of  training  problems 
would  be  minimal,  regardless  of  which  system  was  originally  learned. 

This  was  encouraging  because  one  of  the  major  obstacles  to  Implementing 
advanced  systems  Is  the  resistance  to  change  shown  by  operational 
personnel  who  have  spent  a  number  of  years  becoming  comfortable  with  a 
particular  system. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


Since  both  the  subjective  and  objective  data  reflect  positively  on  EMADS, 
it  seems  safe  to  conclude  that  this  system  can  provide  a  potentially 
useful  supplement  to  advanced  flight  deck  design.  It  should  be  noted, 
however,  that  a  number  of  issues  will  need  to  be  addressed  in  order  to 
refine  the  EMADS  concept  prior  to  actual  implementation. 

There  are  several  human  factors  considerations  to  be  made  that  should 
ultimately  improve  both  performance  and  user  acceptance.  The  first 
involves  the  checklist  capability,  particularly  as  it  relates  to 
abnormal  operations.  In  situations  where  attention  is  to  be  drawn  to  a 
particular  situation  or  corrective  action,  the  cue  that  leads  the  eye  to 
that  information  should  be  the  most  obtrusive  item  on  the  display.  When 
that  information  has  been  conveyed  or  the  appropriate  action  taken,  the 
next  piece  of  information  should  be  emphasized  accordingly. 

For  the  most  part,  checklist  items  for  fault  conditions  are  performed 
serially.  The  present  configuration  of  EMADS  presents  checklist  items 
as  a  list,  with  the  message  on  top  to  be  performed  first,  the  second 
message  to  be  performed  next,  and  so  on.  A  number  of  different  methods 
can  be  employed  to  minimize  operator  confusion  in  performing  checklist 
functions.  The  simplest  method  involves  displaying  only  one  item  at  a 
time  and  when  that  task  has  been  completed,  remove  it  and  display  the 
next  item.  When  all  tasks  have  been  completed,  a  positive  indication  to 
that  effect  should  be  provided  (e.g.  ENGINE  SHUTDOWN  CHECKLIST  COMPLETE). 
Other  methods  include  color  coding,  the  use  of  asterisks(*)  or  checks  (/) 
after  completed  items  (i.e.  prestart  checklist),  manual  cancellation  of 
completed  items  (tasks  that  cannot  be  computer  monitored),  and  flashing 
indicators  adjacent  to  items  not  yet  completed. 

A  careful  analysis  should  be  made  to  determine  the  most  appropriate 
format  for  procedural  checklists  because  delayed  action  could  result  In 
unnecessary  engine  damage. 

The  use  of  color  represents  an  area  where  significant  improvements  can 
be  made  to  the  EMADS  display  format,  clarity  and  readability.  Earlier 
surveys  indicated  that  pilot  acceptance  of  color  coding  would  be  high 
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and  a  number  of  participants  in  this  study  suggested  that  the  effectiveness 
of  EMADS  could  be  increased  if  color  were  appropriately  applied. 


As  mentioned  in  a  previous  section,  a  number  of  the  pilots  who  participated 
stated  that  the  acceptable  command  limits  (-  one  percent  of  the  command 
value)  were  excessive  and  that  they  should  be  reduced  to  one  half  on  one 
percent.  These  limits  should  be  investigated  relative  to  optimum  engine 
operating  conditions  and  adjusted  accordingly. 

Another  area  of  concern  involves  the  genferal  arrangement  of  engine  para¬ 
meter  information  on  the  display  screen.  Conventional  instrument 
configurations  place  the  N-j  dials  at  the  top  of  the  group  for  each 
engine  (with  EGT,  N2,  and  fuel  flow  below).  With  EMADS,  the  N-j 
information  is  always  at  the  bottom  of  the  group,  regardless  of  whether 
the  presentation  is  analog  or  digital.  In  developing  an  optimum  display 
format,  care  should  be  taken  to  avoid  unnecessary  transfer  of  training 
problems.  A  number  of  unnecessary  format  changes  can  be  avoided  if  the 
system  designer  is  aware  of  the  possibility  of  such  problems. 

Finally,  if  EMADS  is  to  be  used  effectively  in  an  advanced  cockpit,  it 
must  be  carefully  integrated  with  other  information  displays.  A  number 
of  display  technologies  are  available  and  under  consideration  for 
advanced  flight  deck  application.  Some  knowledge  should  be  gained  in 
the  areas  of  compatability  requirements,  multifunction  capability, 
backup  display  requirements,  and  panel  space  availability.  Each  of 
these  areas  will  require  investigation  to  determine  the  optimum  control- 
display  arrangement. 

It  is  recommended  that  additional  research  be  done  to  resolve  these 
Issues  as  well  as  those  that  are  exposed  in  the  Interim.  The  ultimate 
goal  is  to  provide  an  efficient,  reliable  engine  monitoring  and  display 
system  that  allows  the  crew  to  effectively  deal  with  normal  as  well  as 
emergency  conditions.  Reliability  Is  extremely  Important  because  the 
Increased  automation  Inherent  In  this  type  of  system  requires  a  high 
degree  of  pilot  confidence  in  the  system's  ability  to  provide  accurate 
Information.  In  addition,  the  novelty  of  the  EMADS  design  relative  to 
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conventional  instruments  may  serve  to  reduce  acceptance  by  operational 
personnel  who  are  not  accustomed  to  evaluating  advanced  concepts.  By 
addressing  each  of  the  aforementioned  issues  and  simplifying  the  display 
design  where  possible,  this  reluctance  should  be  reduced. 
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APPENDIX  A 

COUNTERBALANCING  TABLE 


COMPARATIVE  EVALUATION: 

I  -  CONVENTIONAL  INSTRUMENTS 
COUNTERBALANCING  TABLE 


briefing  guide 

EMADS  CUMPARAUVE  EVALUATION 

Questions  on  reading 

1.  Test  Overview 

a.  Comparative  evaluation:  EMADS  vs.  conventional 

b.  Engine  display  functions  only 

c.  Test  facility 

(1)  Developmental  simulator  (not  training) 

(2)  Relatively  incomplete  cockpit 

(3)  Representative  3-engine  wide-boc|y  transport  (not  necessarily  DC-1U) 

d.  Test  procedure 

(1)  Series  of  partial  flight  profiles:  take-off  and  climb-out 

(2)  Fixed-base  simulation  only 

(3)  Terrain  model  visual  system 

(4)  Test  sequence  (EMADS-conventional ) 

(5)  Test  conditions  will  vary  from  trial  to  trial 

(a)  Take-off  strategy  (rolling/static) 

(b)  Na  limit  value  (high/low) 

(6)  On  some  trials,  simulated  engine  or  system  faults  will  be  introduced 

(a)  Hung  start 

(b)  Hot  start 

(c)  Nj  or  EGT  exceedance 

(d)  Flameout 

(7)  Data  recorded 

(a)  Throttle  setting  accuracy  during  take-off 

(b)  Accuracy  of  aircraft  control 

(c)  Time  required  to  detect  faults  and  initiate  corrective  action 

(d)  Safe  and  accurate  control  of  aircraft 
(b)  Subjective  assessment 

(a)  Comments  during  test  trials 

(b)  Post-test  questionnaire/debriefing 

2.  EMADS  Familiarization  (Video  Tape) 

a.  Overview  of  functional  capabilities  of  EMADS 

b.  During  the  test  only  basic  engine  display  functions  will  be  evaluated 

c.  Some  format  details  on  the  video  tape  are  not  current 

d.  Feel  free  to  ask  questions 

e.  A  detailed  description  of  each  of  the  modes  used  in  the  test  will  be 
provided  in  the  cockpit 
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EMAUS-CONVENTIUNAL  INSTRUMENTS 
COCKPIT  BRIEFING,  PRACTICE  AND  TEST  RUN  PROCEDURE 

1.  Describe  cockpit  configuration  (reiterate). 

a.  Developmental  simulator  (used  as  DC-9,  YC-15  and  DC-1U) 

b.  For  this  study,  represents  3-engine  wi de-body  aircraft 

c.  Not  necessarily  a  DC-10 

1A.  Describe  taping  system  and  ask  permission  to  tape. 

2.  Review  active  displays  and  controls. 

a.  Control  wheel  steering 

b.  Trim  switches 

c.  Fault  detection  switch  (will  describe  later) 

d.  ADI,  HSD,  altimeter,  VSI,  IAS 

e.  Vertical  speed  wheel  and  control 

f.  Master  caution  light 

g.  Overhead  annunciator  lights  (cue  lights) 

h.  APU  isolation  value 

i.  Override/airstart  switch 

j.  Landing  gear  lever 

k.  Flap/slat  controls 

l.  Stabilizer  position  indicator 

m.  Spoilers  (not  used  in  study) 

n.  Parking  brake 

o.  EMADS  control  panel  (EMAUS  portion  only) 

p.  Engine  instruments  (EMADS  or  conventional) 

3.  Simulator  i diosyncracies. 

a.  Rudder  pedal  steering  only 

b.  FMA  similar  to  but  not  Identical  to  that  on  a  DC-1U 

c.  Throttle  response  similar  to  but  not  same  as  DC-1U 

d.  Overspeed  barber  pole  not  appropriate  for  present  simulation 
confi  guration-no  overspeed  warning  will  occur 

4.  (.FOR  EMADS  ONLYj  Review  function  modes. 

a.  Start  cassette  tape  by  cueing  ground  station 

b.  Review  keyboard 

c.  Bring  up  pre-start  checklist  and  complete 

d.  Start  one  engine  to  Illustrate 

(1)  Digital  and  analog  readout  of  engine  parameters 

(2)  Cues  for  pilot  activity 

(3)  Clock 
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4. 


5. 


6. 


7. 


8. 


(Cont ’d) 

e.  T/O  mode  to  illustrate 

(1)  Automatic  and  manual  value  setting 

(2)  Throttle  movement  and  chevron  brightening 

(3)  Allow  pilot  to  set  static  power  1-2  times 

(4)  Illustrate  chevron  dimming  both  above  and  below  command  value 
(b)  II lustrate  exceedance 

f.  Climb  mode  to  illustrate 

(1)  Change  to  N^  values  only 

(2)  Climb  thrust  reduction  to  103.3%  N^ 

(3)  All  engine  instruments  capability 
Review  visual  scene  to  be  used  for  all  flights, 

a.  2400  feet  RVR  and  220  foot  cloud  base 

OR 

700  feet  RVR  and  70  foot  cloud  base 
Review  flight  plan. 

a.  Specific  T/0  strategy  (rolling/static)  and  command  value  for  each 
flight 


b.  Review  each  activity  on  flight  scenario  description 
Co-pilot  activities. 

a.  Call  out  80  kts  and  V  speeds 

b.  Gear  up  on  pilot  command 

c.  Cue  pilot  as  to  flight  management  activities 

(1)  Advise  of  climb  thrust  change  at  2000  feet 

(2)  Adjust  vertical  speed  wheel  at  3000  feet  on  pilot  command 

(3)  Flap  and  slat  retract  at  appropriate  speeds  on  pilot  command 

(4)  [FOR  CONVENTIONAL  ONLYj  Adjust  N^  command  values  for  climb  to 
103.3%  Nj 

Practice  take-offs  and  climb-outs  (2) 

a.  1  static  and  1  rolling  (static  first) 

b.  LFOR  CONVENTIONAL  ONLYj  Conplete  pre-start  checklist  per  reference  sheet 

c.  Start  engines  first  time  only  (Don’t  move  fuel  lever  until  cued) 

d.  Complete  T/0  checklist 

e.  Advise  pilots 

(1)  Should  have  T/0  power  set  by  80  kts 

(2)  Shouldn't  touch  throttles  after  V^ 
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Review  and  define  each  fault  and  action  to  be  taken. 


Fault 

Hung  Start 
(no  ign) 


Hot  Start 


Definition 

EGT  <200o  after  25  sec 


EGT  >750o  for  40  sec 
or  EGT  >250/sec  at 
20  sec  after  fuel  lever 
on 


Corrective  Action 

Detect,  fuel  lever  off, 
after  30  sec.,  pull  st. 
switch,  re-start  engine 

Detect,  fuel  lever  off, 
motor  30  sec,  pull  st. 
switch,  re-start  engine 


l»i  or  EGT 
Exceedance 


Flameout 


Auxiliary 

Faults 


Ni  >116%  Ni 
EGT  >6750 


One  Nj  >ll%  less  than 
other  Ni  with  at  least 
one  hi  at  70% 

Master  caution  light  on 
and  overhead  annunciator 
light  on 

(1)  Select  flap  limit 
override 

(2)  Upper  yaw  damp  inop 
(31  Pneu  temp  high 

(4)  R  windshield  anti-ice 
inop 


Detect,  pull  throttle 
quickly,  back  almost  to 
idle  and  then  back  to 
normal  position. 

Detect,  push  override/ 
airstart  switch  to  on 
position. 

Detect,  depress  overhead 
annunciator  switch. 


(Demonstrate  by  pushing  test 
switch  on  overhead  cue  light 
panel. ) 


Auxiliary  faults  are  configured  on  cue  light  panel  for  this 
stuc|y  and  not  representative  of  any  cockpit. 

Call  out  "detect"  when  detect  switch  is  depressed  and  push  it 
with  force  to  ensure  registration.  (Call  out  is  required  to 
verify  action  because  this  is  a  highly  nonstandard  activity.) 

Advise  pilots  to  depress  detect  switch  as  soon  as  they  sense  a 
problem,  regardless  of  whether  or  not  they  know  what  it  is. 

Be  sure  to  push  fuel  levers  completely  off. 

LFOR  CONVENTIONAL  ONLYj  Point  out  EGT  exceedance  cue  light. 
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10.  Two  practice  runs  (fr2  and  if 3)  with  cues 
a.  Walk  through  each  run  with  pilot  and 

(1)  Advise  as  to  what  faults  will  occur  and  when 

(2)  Advise  as  to  the  appropriate  corrective  action 

11.  Two  practice  runs  (#2  and  #3)  without  cues 

12.  Eight  test  runs 

13.  System  changeover 

14.  Briefing  for  second  configuration 

a.  If  conventional 

(1)  Refer  to  items 
-2P 

-4A,  D,  E,  F 
-BA,  B,  C,  D 
-10 
-11 

(2)  Also  point  out  EGT  exceedance  cue  light 

b.  If  EMADS 

(1)  Refer  to  Items 
-20,  P 

-4A,  B,  C,  D,  E,  F 
-BA,  C,  D 
-10 
-11 


15.  Test  runs 

16.  Debriefing 
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EMAUS-CONVENTIQNAL  ENGINE  INSTRUMENTS 
DE8RIEFING  QUESTIONNAIRE 

1.  Which  system  provides  the  best  overall  status  of  engine  health? 


EMADS 

Si  gnificantly 


EMADS 
Slight  ly 
e 


No  Conventional 

Difference  Slightly 


Conventional 

Significantly 


2.  Which  system  is  most  effective  for  providing  failure 
during  engine  start-up? 

Information 

1  i 

LI  Cl 

I  | 

l_l 

l_l 

1  — 1 

EMADS 

EMADS  No 

Conventional 

Conventional 

Si  gnificantly 

Slightly  Difference 

Slightly 

Significantly 

j.  Which  system  best  facilitates  engine  monitoring  during  start-up  activities? 

Ill  Cl  ill  III  III 


EMAUS 

Significantly 


EMADS  No  Conventional 

Slightly  Difference  Slightly 


Conventional 
Significantly 

te 
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7.  Which 

system  is  most 

effective  in 

guiding  corrective 

action? 

ill 

EMADS 

LI 

EMADS 

III 

No 

III 

Conventional 

Ill 

Conventional 

Significantly 

S11 ghtly 

Difference 

Slightly 

Significantly 

Better 

Better 

Better 

Better 

B.  In  terms 

of  display  clarity,  which  system  is  most 

effective? 

III 

EMADS 

LI 

EMADS 

LI 

No 

LI 

Conventional 

ill  . 

Conventional 

Significantly 

Slightly 

Di fference 

Slightly 

Significantly 

9.  Do  you  forsee  any  problems  In  training  pilots  experienced  with 
conventional  Instruments  In  the  use  of  EMADS: 

Yes _  No _ _  Not  Sure 


If  yes,  explain: 
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instruments  if  their  only  previous  experience  is  with  EMAUS? 

No  Not  Sure 


U 

EMADS 

Significantly 

Better 

Explain: 


EMAUS 

Slightly 

Better 


IJ 

No 

Difference 


Conventional 

Slightly 

Better 


Conventional 

Significantly 

Better 


EMADS 

Significantly 

Better 

Explain: 


EMADS 

Slightly 

Better 


No 

D1 fference 


Conventional 

Slightly 

Better 


LI 

Conventional 

Significantly 

Better 
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11.  In  terms  of  reducing  crew  workload,  which  system  was  most  effective? 


12.  Overall,  which  system  would  you  prefer  in  future  aircraft? 
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APPENDIX  D 
ANALYSIS  OF  VARIANCE 
SUMMARY  TABLES 
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IN-FLIGHT:  FAULT  MONITORING 
DETECTION  TIMES 
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AVERAGE  DEVIATIONS 


QUESTIONNAIRE  RESPONSE 


I 


Question:  Which  system  is  most  effective  for  providing  failure  information 

during  engine  start  up? 


\\ 

Response 


Number  of  Respondents 


EMADS  Significantly  Better 
EMAUS  Slightly  Better 

Comments  (paraphrased) 

The  presentation  of  engine  parameters  with 
EMAUS  is  easier  to  monitor  for  abnormalities 

EMADS  gives  more  obtrusive  indication  of  fault 
information 


& 

4 

Number  of  Respondents 


4 

4 


EMAUS  provides  a  useful  checklist  capability  3 

Question:  Which  system  best  facilitates  engine: monitoring  during  start-up 

activities 


Response 


Number  of  Respondents 


EMAUS  Significantly  Better  3 

EMADS  Slightly  Better  4 

No  Preference  1 

No  Response  1 


Comments  (paraphased)  Number  of  Respondents 

Pertinent  information  is  in  closer  proximity 

with  EMADS  i 

Better  simplicity  of  presentation/logic  with  EMADS  1 

Would  like  to  have  view  of  all  engine  parameters 

when  in  start  mode,  as  with  conventional  dials/tapes.  1 


X 

Question:  Which  system  best  facilitates  the 

*>>. 

power? 

V\;  • 

•>.*  v.v; 

• 

lS.  ^ 

Response 

a 

EMADS  Significantly  Better 

m 

EMADS  Slightly  Better 

.W.s 

if.w 

!•*,*«*.* 

Comments  (paraphrased) 

EMADS  provides  a  more  noticeable  target  with 

Number  of  Respondents 


chevron  intensity  change 

EMADS  display  appears  more  precise  and  easily 
readable 

Small  digital  figures  in  EMADS  somewhat  difficult 
to  read 

EMADS  tolerance  limits  too  large 
Question:  Which  system  best  facilitates  the  monitoring  of  in-flight  faults? 


b 

l 


Number  of  Respondents 

6 

3 

1 

1 


Response 

EMADS  Significantly  Better 
EMADS  Slightly  Better 
Conventional  Slightly  Better 
No  Response 

Comments  (paraphrased) 

EMADS  provides  better  attention  getting  cues 
Color  would  be  a  useful  addition  to  EMADS 
Less  scanning  required  with  EMADS  because 

of  automatic  fault  monitoring  capability  1 

Question:  Which  system  is  most  effective  in  guiding  corrective  action? 

Response  Number  of  Respondents. 

EMADS  Significantly  Better 
EMADS  Slightly  Better 


Number  of  Respondents 

5 
2 
1 
1 

Number  of  Respondents 

6 
2 


b 

4 


Comments  (paraphrased) 

EMADS  provides  checklist  presentation 

EMADS  would  be  more  effective  if  color 
was  enployed 


Number  of  Respondents 
4 


vv 
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Question: 


In  terns  of  display  clarity,  which  system  is  most  effective? 


R 


V.V' 


Response 

EMAUS  Significantly  Better 
EMAUS  Slightly  Better 


Number  of  Respondents 


Comments  (paraphrased) 


Number  of  Respondents 


EMAUS  is  easier  to  read  2 

The  use  of  color  would  enhance  the  clarity  of  EMADS  1 

By  providing  only  necessary  information,  EMADS 

becomes  the  more  desirable  display  1 

Question:  In  terms  of  reducing  crew  workload,  which  system  was  most 

effective? 


Response 


Humber  of  Respondents 


EMADS  Significantly  Better 
EMADS  Slightly  Better 
No  Difference 


Comments  (paraphrased) 


Number  of  Respondents 


The  automation  that  Is  part  of  EMADS  is  a 
significant  worksaving  element 

EMADS  does  not  require  a  high  vigilance  level 

More  Information  can  be  obtained  In  a  shorter 
period  of  time  with  EMAUS 


Question: 


Overall,  which  system  would  you  prefer  in  future  aircraft? 


EMADS 

Conventional 


Comments  (paraphrased) 

Display  and  software  provide  for  faster 
retrieval  of  necessary  information 


EMADS  provide  a  better  attention 

getting  capability  2 


EMADS  provides  all  necessary  information 

in  the  same  location  2 


EMADS  should  be  reformatted  to  provide  more 
readable  and  action  oriented  messages 

The  EMADS  concept  Is  excellent 


The  EMADS  display  Is  easier  to  read 


A  lower  level  of  vigilance  Is  required  with  EMADS 


1 

1 

1 

1 


